Weak Convergence and Fluid Limits in Optimal Time-to-Empty Queueing Control Problems

We consider a simple class of controlled queue length processes in the fluid rescaling, X(t) = 1 n X(nt). Consider the optimal control problem for X(t) which consists of minimizing the integral of an undiscounted positive running cost until the first time that X = 0. Our main result uses weak convergence ideas to show that the optimal value functions V n of the stochastic control problems for X(t) converge (as n → ∞) to the optimal value V of the fluid limit control problem. This requires certain equicontinuity and boundedness hypotheses on {V }. We observe that these are essentially the same hypotheses that would be needed for the Barles-Perthame approach in terms of semicontinuous viscosity solutions. Sufficient conditions for these equicontinuity and boundedness properties are discussed.